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January ' 6, 1967 

There has been much discussion, in recent months, of the 
fact that systems analysis or operational analysis is an effective way 
to study complex systems. [9] If this is true, then we should 
certainly attempt to apply these techniques to one of the most vital, 
complex systems of our society, the urban school district. To give 
impetus to such analysis, we will, in this paper, review what is 
meant by operational analysis and explore the first steps that would 
be required to apply it to the study of urban school systems. 

What Operational Analysis Can Do 

Early prognostications of the value of a new technology tend to 
be over -optimistic, and so it might be wise to remind ourselves just 
what we can expect from any analytic tool. Tools of analysis, when 
completely effective, are methods of predicting what will happen if 
we take certain courses of action. Theories of physics tell us what 
will happen if we apply more than a critical amount of thrust to an 
object in the vertical direction. that object will go into orbit. We 
need not choose the course of action studied. The analysis makes a 
prediction, it does not issue commands. In the case of human 





endeavors, we also would like to find the course of action that will 
be "best" for the public being served by the endeavor. In simple 
cases (such as inventory control), it is possible to find by analysis 
the "best" course of action. However, as we study more and more 
complex systems, we find that the selection of the optimum or best 
course of action becomes nearly impossible, even with the aid of 
mathematical tools. 

Complex systems involve many difficulties which do not exist 
in the lov/er level studies. First, it is often very difficult, if not 
impossible, to identify a single goal or objective. Many groups are 
involved in complex human organizations and each group tends to 
have a different set of goals which cannot be reconciled exactly with 
those of others. Since there is no clear cut goal, it is difficult to 
define what "best" means. Furthermore, the complexity of the system 
usually means that no concise mathematical formulation of a model 
can be obtained. And if a mathematical model cannot be developed, 
then it is extremely difficult to develop techniques for finding the 
"best" solution, even if goals can be defined. 

Thus, although operational analysis might be extremely useful 
in providing us with predictions of what can happen if a course of 
action is taken, it will not in general, tell us what the best course of 
action is. The ultimate decision will still be made by appropriate 
elected or appointed authorities. But having predictions in the form 






















of quantitative estimates of what may happen is much better than 
the groping studies now made in support of decisions on educational 
programs.. 

The Natm;e of Operational Analysis 

analysis is the quantitative study of a system. 

The word "system” in itself has no particular meaning* Almost 
anything can be abstracted as a system, from the universe as a 
whole to an assemply of tinker toys. The system is that which 
a particular group has chosen to study. 

An operational analysis effort involves several identifiable, 
but complex, steps. The first three steps involve: defining the 
boundaries of the system under study, denoting the major subsystems 
and the way in which they interact and from these efforts, 
identifying the important variables. Since the research group must 
define the system under study, the first step is to identify the 
boimdaries of the system which will be studied; that is, to identify 
v/hat processes and phenomena are to be within the system (and 
studied in detail), and what processes and phenomena will be 
considered to be outside the system. Analysts tend to choose rather 
wide boundaries at first and narrow them down as the understanding 
of the system evolves. In other words, the boundaries chosen at 
first may be changed as the analysis proceeds in order either to 
incorporate better understanding of the system or to focus on a 
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critical subsystem. At any time in the analysis, however, some 
set of boundaries is defined. 

If we take as our problem area an urban school district, 
the choice of boundaries is extremely difficult. For example, 
it is obvious that the group of people called taxpayers have a 
significant effect on the school district; they provide a large por- 
tion of its financial resources. On the other hand, we would not 
generally include the taxpayer as a part of the urban school 
district. For the purposes of the study we wish to outline below, 
however, let us broaden our scope so that we are, in effect, 
studying the educational system of a region. Therefore, we might 
legitimately include the taxpaying group as a source of funds, 
just as we might include the shareholders as a part cf a corporate 
organization. Obviously the setting of boundaries to define a 
system as complex as a regional educational system requires care; 
if the system defined is too large analysis may be infeasible, if 
too small important phenomena may be excluded. Even geographic 
limits are difficult to define since, for example, teachers may be 
recruited from outside the region under study. 

Assuming, however, that boundaries have been set, the next 
step is to identify the basic phenomena within the system. There 
are several types of phenomena which must be integrated as the 
analysis proceeds. One type includes the ongoing processes. In 
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an urban school system, examples of these would be: teaching 
(of many different kinds), construction of buildings, hiring of teachers, 
training of staff, etc. The second type includes the key decisions 
that are made throughout the system, such as: allocation of basic 
resources, decisions about personnel (hiring, assignments), 
decisions about curriculum., decisions about the location of schools, 
etc. The third phenomena is flow. In a school system there are 
flows of people from grade to grade, from the region, etc. There 
are flows of money or financial resources, flows of inform.ation 
and flows of materials and supplies. Below we will discuss the 
operational method of studying a system which incorporates all 
three of these types of phenomena. 

Use of a Model 

In operational analysis, the major methodological concept 
is to develop a model which can be used to study the system. [ 1] 

After the subsystems, decisions and flows are identified, model 
development can begin. The function of a model is well known 
in the physical sciences and has led to many cf the advances in these 
disciplines. A model is even more important in the study of a 
complex system, such as an urban school district, since it is ex- 
tremely QKperisiveiand^difificaait'to run e'xperihients on the system itself. It 
is therefore important to understand the nature of a model. 
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A model is a simulation or an analog of the system under 
study, (We will, however, use the word ’’simulation” in a 
narrower sense subsequently). There are several kinds of models, 
the most obvious being a physical model. For example, a model 
of an aircraft in a wind tunnel is a physical representation of key 
cLspects of the real aircraft. The model is used to determine 
aerodynamic characteristics and to predict how the real plane 
would perform under various circumstances of flight. Physical 
models are not particularly useful in systems wher© the flow of 
information is a major phenomena as it is, in a school system, A 
second class of model is the symoolic -mathematical model, [8] 
This is the form of model which has been so successful in 
physical sciences. In this form cf model we represent the system 
under study by two types of symbols. The first represents the 
values of variables describing the parts of the system*, the second 
represent the relationships between these parts. The first type 
of symbol represents quantities, which, at least in principle, can 
be measured. The second or operational symbols, correspond 
to real-world processes or interactions between parts of the 
system. In using the model for analysis these operational symbols 
define logical manipulations of the descriptive variables. In 
other words, the operational symbols tell how the variables should 
change as a function cf each other. 



7 . 



The besLuty of the symbolic model is that, in sonie cases, 
we can find a mathematical structure which will help us manipulate 
the symbols so as to. ^ ' - 

make predictions very efficiently. If such a mathematical structure 
can be found or developed, then, by the manipulation of the symbols, 
we can develop formulas which will allow us to predict what would 
occur under a wide variety of circumstances. In fact, the existence 
of a mathematical structure often lets us go further. It lets us 
find the "best" solution (providing we can define a measure of 
"goodness"). It also lets us explore hoxv performance will change 
as we take courses of action other than the "best" one. Unfortunately, 
however, it is very unlikely that we can find a mathematical structure 
which will permit us to manipulate a symbolic mcdel of a situation 
as complex as a school district. Mathematics, advanced as it 
is, has not developed techniques which permit the manipulation of 
the thousands of highly interacting variables required to model 
a school district. 

There is a third kind of a model, usually called computer 
simulation, which is appropriate for complex socio-economic 
system studies. [ 5] The variables of the system are represented 
by numbers within the computer memory. The dynamic processes 
are represented by a flow chart and dynamically modelled by a 
computer program which determines how the variables will change 
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as tim© procseds. Aftsr dofining th6 variablss and th© program 
or procedure for their change, we can actually execute the model 
on a computer. That is, the simulator can be run. The result 
of running the simulator is a prediction of what will happen under 
the circumstances built into the model. Other runs can be made 
to make predictions under other circumstances. Thus, to study 
an urban school district through operational analysis techniques, 
a computer simulation model w ould be developed. 

To review the discussion thus far, the steps of the analysis 
are: defining the system boundaries; identifying the major 
processes, flows, and decisions; isolating the variables and relation- 
ships implied by these processes, and developing a computer 
simulation model which represents or ^ odels the sytem. 

In the final step of analysis the simulation model is used 
to study how the system will behave as various courses of action 
are adopted. For example, we could study how the system would 
behave if we reduced the student -to -teacher ratio, if we provide 
extensive remedial courses (for example, in reading or mathemaUcs), 
if we provide additional staff development, if we change the 
management organization of a school district, or for other kinos 
of action that could be taken. 

Problems with Operational Analysis in the Context 

There are two major difficulties in developing a simulation 
model which will be sufficiently representative of a real school 
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district to be useful. The first problem is to obtain measures of 
performance based on the system goals. This problem cf course 
does not result simply from an attempt to perform operational 
analysis. The management control of school districts and the 



allocation of resources to a school system relative to other municipal 
functions both suffer from the fact that we have no good way of 
measuring the performance cf a school district. Certainly, if we 
are going to analyze the system in order to choose better courses 
of action, we have to know what "better" means. Below we will 



propose some solutions to the problem cf developing performance 
measures for a school district, and it should be clear that this is 
a step vital to the success of any analysis. 

The second problem is that the model must represent the 
behavior of groups cf people. It is possible to develop computer 
simulations v/hich represent at least some phases of behavior. 
However, it is difficult. It is even more difficult to prove that 
these models are valid; i. e. , good representations. A school 
system model will, for example, have to represent the behavior 



of students as they are taught under diverse situations. It will have 
to represent the behavior of taxpayers in regard to their willing- 



ness to support schools and the behavior of industrial management 
in regard to the employment of persons of various skills. It will also 
have to represent the behavior of teachers under various conditions of 
pay, management and availability of support. 
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We will discuss below this question of modelling behavior. 

It is difficult but an important step, if we are to succeed with the 
operational analysis. 

Some Considerations in Modelling Complex Systems 

An important decision which the systems analyst has to 
make in developing a model is to determine the level of aggregation. 
In other words, he must determine to what level of detail he is 
going to represent the real world. He could on one hand represent 
every student and every teacher every hour of the day, and thereby 
model, at a very exhaustive level of detail, the teaching process 
and all of its implications. At the other extreme, we could represent 
the annual activities of a school district and summarize the teaching 
process, the construction process, etc. , in aggregate terms 
summarizing what is accomplished over a year. The choice of 
level of detail depends upon the purposes of the model and may 
change as the analysis develops. The analyst may expand the 
detail in part of a model in order to get a better representation of 
the real process, or he may aggregate a detailed part, if he finds 
that the detail does not contribute to the understanding of the system. 
In the discussion below, we will suggest some choices as to level 
of detail based on a preliminary, intuitive feeling as to what makes 
the difference in the system. 
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At whatever level of detail, the model usually can be 
developed in terms of terms of flows, processes, and decisions 
as we mentioned above. A process is an identifiable collection of 
activities which has inputs ard outputs. The process is modelled 
by defining the outputs (at each interval of time to be represented) 
as a function of (that is, related to) the inputs at all previous 
times. For example, we can consider the teaching of first grade 
in a school district as a process. If we are working on a monthly 
basis, then we would want to represent the state of learning 
(however we may measure it) of the students at the end of each month 
as a function both of the resources which were applied during that 
month and the capabilities of the students at the beginning of the 
month. The inputs are the students with a certain level of learning 
and the resources (the teachers, the materials, the building, etc. ). 
The output is the level of learning of the students at the end of the 
month. For every process in the system, we will need to develop 
these output-input relationships. The inputs and outputs can consist 
of several types of variables representing the different flows: people, 
information, materials, and financial resources. Consider a variable 
which numerically represents, for example, students' ability to 
read on some scale. This variable would be an output of the teaching 
process. Its value at the beginning of the month would be an input 
variable. The teaching hours applied to the student group might be 
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anoth©r variable, in this case an input variable. Thus, we can 
say that the process is modelled when we can define the value of the 
output variables as a function of the values of the input variables at 
earlier times. In defining" the system, we must also, of course, 
indicate which output variables from one process become the input 
variables to another process. The interconnection of variables in 

this manner represents the flows in the system. 

Finally, there will be decision processes. Decision processes 
are processes like those we have just described^ thsre will be 
output variables which are functions of input variables. The only 
difference is that the process represents the intellectual behavior 
of some decision maker and the output and input variables are always 
information variables (and not material or other flows). 

A system model can thus be portrayed as a series of 
processes — decision and operational -- interconnected by command 
variables. It is customary to represent each process by a box in 
a diagram and to show by arrows which output variables become 
input variables to other processes. Below, we will attempt to develop 
such diagrams for an urban school district. 

An Acfcrrecfate Model of an Urban School District and Its En vironment 

Figure 1 is an effort to summarize by a diagram a system 
model, at a very aggregate level, of an urban public educational system. 
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The level of aggregation varies, being more detailed for the 
educational sectors and less for other sectors of the total socio- 
economic environment, (The subsystem enclosed in a dashed line 
will be developed in more detail below). 

In Figure 1, the "p” arrows show the flow of people through 
the system. The small circles are points where decisions affect 
flows. These are connected to larger circles which represent a 
decision process ("d” flow). For example, to the left of the diagram 
we show parents deciding whether to send children to public or 
private school. They base this decision on information which they 
obtain about schools and about opportunities for work and for higher 
education after school. They also base this decision on the financial 
support available, (Parents are generally also part of the labor 
force and thus have income, of course). 

The "i” arrows show the flow of information from decision 
makers to the decision points. As another example, to the right 
of the diagram, industrial management controls the flow of people 
from the educational subsystems into job positions in the productive 
processes. 

The ”f ” arrows show the flow of financial resources (money) 
throughout the system. In a sense, money is a form of information, 
but a special form in that it represents value in the socio-economic 
context. Note particularly the flow of money from the productive 
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activity in the form of taxes through government agencies back to 
the school system. This, of course, is a summary for many 
complex flows of tax revenue received by Federal, State and Local 
governments and allocated to the schools. 

In Figure 1 we have summarized the information subsystem 
as one process. Actually it would have to be studied in more 
detail to represent what information each decision maker receives 
and uses to make his decisions. 

A Diaqramatic Representation of the School District 

Figure 2 summarizes a more detailed model of the school 
district itself; the subsystem in the dashed line in Figure 1. 

(Recall that the boxes represent computational models of processes 
and the arrows represent interconnections of variables which 
model flows). The similarity between the structure of the decision 
processes in Figure 2 and anoigaoization’s chart is not accidental. 
An organization chairt is an effort to show how responsibility for 
making decisions is delegated^ Here we have shown not only the 
flow cf decisions but the flow of financial resources, (for example, 
in the form of budget allocations) the flow of students (”s”) and 
teachers ("t”) and also the flow of information (”i"). This sub- 
system might have to be detailed even further to obtain a good 
representation of a school district. 









V 

'V' 







16 . 



•o 

c 



iV2 

P 



o 

o 



0 

oO 



d 






c^ 



>si 

Oo 



V, 

u: 



'j- 

M 

-P £ 

i2 









CO 






•I 

VI 

«« 

V 



•K 






0 0 



'-> 




*-n 

s 



^5: 






‘-S- ^ ^ 



s« s ' 



^«3 



S^ 

o -~3C ^ 



Hk 

s. 



^•c, 



s S E « V ^ 
^ S |-? 

Ca^ ^ ^ 

-in- 



, > ' 
•o-U'-i 



O 

ERIC 


















... , 



o 

RIC 



17. 



A Simulation Model 



The diagrammatic model shown in Figure 2 is intended 



to convey the nature of the interactions assumed to be operative. 



It is not, however, a representation of a simulation. A simulation 



that represents the structure shown in Figure 2 can be developed. 



Recall that in a simulation model the key variables are represented by 



numerical quantities. Each of these numerical quantities will, of course, 



have a name which will identify it. Secondly, in a simulation model 



we have a procedure (which we will show as a flow chart) which 



describes how to manipulate the variables. Figure 3 is a flow 



chart for a model which represents an aggregated school district 



on a year by year basis. Some of the variables are named at the 



top of Figure 3 at the point in the simulation procedure where their 



values are calculated for the current (simulated) time period. 



The meaning of the boxes and the arrows in Figure 3 is somewhat 



different than in Figures 1 and 2. Here each box represents the 



sequence of arithmetic and io0cal steps which are to be carried 



out on the variables. These steps correspond to a model of the 



relationships supposed to exist in one of the processes of the system 



model and, hence, in the real situation. The arrows in the flow 



chart represent the sequence in which the computations are to take 



place. Since v/e are representing an entire year in one step, the 



sequence in which computations are made does not bear any particular 
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relationship to the sequence in v/hich things occur in the real world. 

It is the nature of computer simulation that they often represent 
sequentially processes that actually take place in parallel. This 
timing discrepancy does not invalidate the model. If the model is 
properly designed, useful studies can be made of long term phenomena. 
Studies of short term phenomena or of the detailed operation of the 
system would require simulations with a shorter basic time interval. 

This model, as any model, must be oriented toward 
ansv/ering specific questions; it must have a purpose. A model which 
can answer any question about the system would be far too complex. 

The model similar to that summarized in Figure 3 has been developed 
at the University of Pennsylvania. [ 10] It is oriented toward 
answering those questions which arise in managing an entire school 
district; that is, toward studying the phenomena from the point 
of view of top management, the Board of Education and the 
Superintendent, For this reason, the model is quite aggregate and 
does not represent many of the detailed processes (which would be 
needed if we were to answer questions having to do with the operation 
at the level of the principal or the teacher). Also, the model is 
oriented toward answering questions about resource allocation 



rather than questions about personnel selection or system organization. 
These other questions are vital, of course, and the overall model 
based on Figure 2 will have to be developed to study organizational 
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problems, which are so difficult in urban school systems and for 
studying other management decisions. 

Consider Figure 3 again. It summarizes the various 
computations which have to be performed (in sequence) to represent 
Figure 2. First there is a process which represents the birth of 
people and their growth through to preschool age, A second part of 
this computation represents the parent decision to send their child 
to private or public school. Thus, the output of this computation 
is an estimate of a number of pupils of various ages to be educated 

in a public school system of the region. 

The next computation determines the characteristics of 
these pupils. (In real life, of course, the characteristics would 
develop at birth and in the early years of growth). This calculation 
distributes the characteristics throughout the student population 
as we hypothesize they would actually be distributed. At this 
point in the computation we have estimated how many students are 
physically handicapped, how many might have exceptional ability, 

and so on. 

Next we simulate the process by wliich management of a 
school district estimates the funds it will have available from norma^ 
sources and can obtain by special efforts, (passive or aggressive 
managements can be simulated). Having the funds available and 
knowing the characteristics of the student body and also having 
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the previous yestr's operational results, the simulator models the 
process of management making allocation decisions, decisions 
about how to operate the school system for the coming year. This 
includes decisions to hire additional teachers, decisions to create 
additional space, and the allocation of funds between the various 
programs and activities (such as teacher development, remedial 
reading, the normal grade teaching, etc. ). Next it simulates the 
process by which teachers are recruited and hired. (Hires are 
not necessarily equal to the number sought by the decision maker, 
depending on salary decisions and the market for teaching personnel). 
A detailed model would also simulate the characteristics of the 
teachers in terms of their abiUty, skills and effectivensss ( a very 
difficult part of the model to develop). Next, the simulation 
computes the actual results of facility construction (which may of 
course lag behind or cost more than that scheduled by the decision 
makers). The result of this step would be calculations of space 
available for teaching. Alternatives, such as renting rather than 
building could be studied. 

The next computation in Figure 3 represents the imparting 
of skills, knowledge, motivation, and learning ability to students 
by teachers. This, of course, is a simulation which will require 
intensive study to develop, as it models both teacher and student 
behavior. The result of this step is some increase in the knowledge 






.’'Jncww 



22 . 

and potential of pupils. This computation also estimates the effect 
of the teaching process and of environmental factors on admin- 
istrative performance such as attendance, dropouts, perhaps 
disciplinary case rate, etc. 

Next we can simulate the process of testing which will 
produce test results to be incorporated into measures cf performance 
The results of the tests will, of course, depend both on the nature 
of the tests and on the potential of the students as computed above. 

The simulation proceeds by performing cost accounting 
calculations to determine the cost of the simulated processes: 
recruiting, construction, teaching, testing, etc. Next an estimate 
cf actual funds available is fed into the computation. This model 
does not include the simulation of the funding part of the system 
and so a pre-computed estimate is substituted. In a complete 
model, available funds would be computed as a function of other 
community processes. 

A final accounting produces a financial statement for the 
system. Thus, the simulation gives us two critical outputs — a 
measure of performance of the system as it would be indicated by 
tests and other measures, and a financial statement. It also 
gives many intermediate outputs, such as estimates of teacher per- 
formance, and operating statistics such as the student per teacher 
ratio, and square feet of space per student, etc. 
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Obviously the most difficult sub-model to simulate is 



the teaching -learning process. If a start is made in the devel- 
opment of such a simulation it will have extremely beneficial effects 



in pinpointing what we have yet to learn about the learning process. 



Such a start can be made by reviewing past studies of the learning 



process, by new experimentation and by formalizing the educator's 



intuitive understanding of the learning process. 



Measuring Performance 



In Figures 2 and 3 provision is made to simulate th^^ process 
of testing students. This part of the model is intended to provide a 
measure of performance for the system. The philosophy adopted 



here is the following; 

The ultimate measure of an urban school district is the 



performance of its graduates in the community. Thus, the system 
must develop people who have many individual and community 



characteristics such as: being a good citizen, holding a job. 



contributing to the society, being happy, being a g^od neighbor, etc. 
A secondary measure of some of these is the general well-being 



of the community as measured, for example, by the gross regional 
product. These measures have a very long response time. We 
cannot use them to make decisions about what to do next year 



in the school. There must be some intermediate measure of 



performance which, on one hand, takes at most a few weeks 
















p^f^iJWPp j kw, «ph' j li'y l}, 









24 . 

(in real life) to determine, and which, on the other hand, is a 
good indicator of the more ultimate measures of performance. 

For this purpose we conceive that each class or group of students 
is periodically (yearly or more often) given a series of tests. 

The weighted sum of test scores and of other measurable character- 
istics (e. g. , attendance) then becomes a measure of performance. 
These tests can include performance in gaming situations, and need 
not all be paper and pencil tests. Thus, they can be made more 
representative of the ultimate measures of performance than our 
present tests. (It is true that teachers will then try to teach so as 
to have students do well on the tests. Tentatively, it can be argued 
that, if the tests are well conceived, that this would be a good 
thing. It is exactly analogous to the concept of profit^ the immediate 
goal for which every businessman strives and which, on the other 
hand, is presumed to be a good thing in the long run for the community. 

The simulation permits the study of the effect of management 
decisions on both the performance attained as indicated by the 
intermediate measures and costs. Cost -performance tradeoffs can 

therefore be analyzed. 

As the simulation is developed, a model which relates the 
educational process to the community may evolve. Then ultimate 
measure may be estimated directly. Future operational analyses 
of the school-community interaction might explore other measures 
























of performance; for example, a concept of value-added. It might 
be possible to estimate the economic value added to the commumty 
by an educational process of a given quality. As the quality 
increases, the value-added should increase* Everyone agrees that 
the level of industrial and economic activity of a community must 
be higher if the children of the community have gone through a school 
system than if they have not. The value-added concept would be an 
attempt to formalize this intuitive feeling. 

Representing Behavior 

Given the success in predicting elections from early returns, 
we can now say that at least some forms of behavior can be 
predicted. [4] People often do not like to feel that behavior is 
predictable, but when we talk about the behavior of groups of people 
(and not individuals) it turns out to be quite predictable. However, 
behavior is still very complex. Therefore, simple mathematical 
models or simple qualitative statements are not successful in defining 
behavior. However, a computer model can often represent all of 
the complexities involved* 

In addition to the election prediction scheme, we will mention 
three other experiments which show that behavior can be predicted. 
Cyert at Carnegie Institute of Technology has programmed a 
computer to perform the same decision making process that a trust 
officer performs in selecting stocks for a portfolio. [3] The 
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computer did at least as well as the trust officer, and in general, 
chose the same stocks as he did. Greenlaw at Penn State has 
programmed a computer to perform the same analysis that an 
industrial psychologist makes to recommend or not recommend 
people for specific jobs. [ 7] The computer analyzed the raw test 
scores, just as the psychologist does to develop his recommendations. 
There is an extensive simulation model developed by Amstutz of 
MIT which represents the pharmaceutical market including physician 
prescription-v/riting behavior. He has also represented retail 
consumer behavior by simulation [2] 

In each of these cases extensive study was required of actual 
behavior to develop the model. However, each of these models has 
been validated by experiments in the real world and their success 
indicates that the technique is useful. The operational analysis 
of a school district would be considerably advanced if we would begin 
to develop simulation models based on studies of the behavior of 
students, teachers, parents, and others directly affected as quickly 
as possible. 

The Final Steps of Analysis 

Once a model is operating, we would like to determine its 
validity. That is, we would like to be able to show that the outcomes 
predicted by the model are indeed those that would occur in actual 
life. It is extremely difficult to validate a very complex model 
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for a system in which we cannot run experiments and within 
which the time responses are very slow. If we have to wait 10 
years to determine the effect of a new teaching technique, it becomes 
discouraging to try to validate a model which has predicted the 
outcome of the introduction of that teaching technique. One partial 
validation procedure is to be sure that the model represents the 
behavior of the system over the past few years. A second validation 
technique is to be sure that each submodel, in particular, the 
behavioral submodels, represent the phenomena they are intended 
to represent as accurately as possible. However, in operational 
analysis of complex systems one will always be using models 
which are not completely validated. The only justification for the 
use of such models is that they offer more precise and compre* 
hensive insights into the system than does any other approach as 
yet devised. Further, since the model would incorporate the 
intuitive notions of educators and others connected with the system, 
it should, in no case, make worse predictions that they could. 

The ultimate test of the sophistication of a model might be 
this: Could it represent the behavior of the various decision makers 
and the various processes sufficiently well so that we could test 
such dramatic proposals as those made by Friedman [6] and 
others to, in effect, do away with a public school system and give 
every parent a subsidy for sending his children to an approved school. 
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We would hope, ultimately, to use models to study both extreme 
and novel proposals to evaluate whether the quality of education 
would increase or decrease and to evaluate v/hether the availability 
of education would be restricted or expanded. 

Once the model is validated it would be used to study the 
system. Its use would depend on whether one was a researcher or 
a manager. The researchers would use a model to explore the 
nature of the interactions involved. They might study questions 
such as: What happens to the school system if we increase the total 
resources (particularly financial resources) available? What 
happens to its performance, if a sufficient supply of qualified 
teachers cannot be obtained? Can performance be increased through 
the use of non-certified personnel or through the use of computer- 
aided instruction? What is the interaction between space allocations 
and teaching performance? A whole host of such questions about 
the school system phenomena v/ill occur to any reader. Through 
a series of experiments on the simulation model these phenomena 
can be studied. Interactions that appear particularly interesting 
might then be validated with tests in actual school districts. 

A manager would want to use the simulation model as a tool 
for exploring specific policy actions under consideration. For 
example, if a school district proposes to introduce a major remedial 
reading program, the simulator might be used to evaluate the 
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pi'obability thEt th© program will indeGd improv© r©ading p©r- 
formanc©, that it will improv© g©n©ral l©arning p©rformajic© and 
also to ©stimat© th© total cost impact of th© program on th© system, 



not only for th© current year but for future years. 



Summary 



Operational analysis can be an effective tool for gaining 
understanding of a complex system such as an urban school district. 
The steps required to implement the analysis involved: 

- Identification of the system under study, 

- Identification of the subsystems, the processes, flows 
and decisions, 

- The development of a simulation model, which includes 
the development cf models of behavioral and performance 
subsystems, 

- The validation of the model by tests in schools, 

- The use of this model to explore the system characteristics, 



Such an operational analysis is expensive. It requires very 
talented people, skilled in operational analysis methodology, in 
education and in the management of school systems. A team effort 
is indicated. Extensive data gathering is required to obtain the 
parameters necessary for the model and to validate the key submodels. 
Nevertheless, the cost of such analysis -- possibly runmng into 



'mtm 















30 . 



millions of dollars — may be trivial compared to the benefits 
derived through the improved understanding of what makes a 
difference in the performance of an urban school district. 
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